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ABSTRACT 



One of the ways in which tutors can help students improve 
their academic writing is to give them positive and constructive feedback on 
their work. D. Hounsell, however, suggests that written comments may fail to 
connect because tutors ' and students ' perceptions of marking criteria can be 
very different. This paper reports on two research studies, both concerned 
with how lecturers can make their written feedback effective. The paper 
states that the first study (Norton and Norton, 2001) reports on the use of 
an essay feedback checklist as a means of more effectively targeting written 
feedback, while the second study, an earlier piece of research by L.S. Norton 
(1997) , reports on how feedback was perceived by students and its effect on 
their motivation to improve their academic writing in subsequent essays. It 
explains that the first study was carried out in a psychology department of a 
university college in England with 61 first-year students and 65 third-year 
students (Figure 1 of 2 contains the essay feedback checklist) . It also 
explains that the second study was conducted with 47 third-year psychology 
students (in stage 1) and continued (in stage 2) with 40 of the original 
group. The paper concludes that using the essay feedback checklist is 
worthwhile, and students are strongly affected by the grade they receive. 
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Research context 

One of the ways in which tutors can help students improve their academic writing is to give 
them positive and constructive feedback on their work. In the context of coursework essays, 
tutor feedback in the form of comments and a grade is often the only help students will get 
(Hartley, 1983) so it is important that such feedback is useful. Unfortunately, this is not 
always the case. Hounsell (1987) stated that feedback on coursework essays appears to be 'a 
central assessment activity but a peripheral pedagogical one.’ Sometimes though, even when 
good quality feedback is given, it appears not to be acted on by students in their next essays. 
Hounsell suggests that written comments may fail to connect, because tutors’ and students’ 
perceptions of marking criteria can be very different. Recent research in this area would 
appear to confirm this (see Longhurst & Norton, 1997; Norton, 1990; Norton, Horn & 
Thomas, 1997). 

In the course of the conference presentation, which is in two parts, two research studies will 
be reported, both concerned with how as lecturers we can make our written feedback 
effective. The first study (Norton & Norton, 2001) reports on the use of an essay feedback 
checklist as a means of more effectively targeting written feedback. The second study, an 
earlier piece of research by Norton (1997), reports on how feedback was perceived by 
students and its effect on their motivation to improve their academic writing in subsequent 
essays. 
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Study 1 

The essay feedback checklist: How can it help students improve their academic writing? 
Norton, L.S. & Norton, J.C.W. (2001) 

The research was carried out in a psychology department of a university college in England in 
which both students and their tutors were asked to complete an essay feedback checklist. The 
checklist listed 8 departmental assessment criteria which were routinely used for the marking 
of psychology essays and then asked students to tick whether they thought they had done each 
one by answering ‘yes ' or 'partially' ox 'no ’. (See a copy of the checklist in Figure 1). 
Students were asked to complete the essay feedback checklist before submitting their essay 
and attach it to the essay when they handed it in. The tutors were asked to use the same 
checklist as the students, when marking their essays. First year students and third year 
students were asked to take part in the study to represent those at the beginning and nearly the 
end of a degree course. Completed checklists were attached to essays by 6 1 first year students 
(about 25%), and from 65 third year students (75%) taking a specific module ‘Counselling 
Psychology.’ Five first year lecturers and the two lecturers who delivered the counselling 
psychology module also agreed to take part in the research. They were asked to complete the 
checklists after having marked the essay but before assigning the final grade. In addition, 
four of the first year lecturers and one of the third year lecturers agreed to be interviewed 
about the process. 
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The two main aims of this research 

1 . To see whether there were any mismatches between students and their tutors in order to 
consider whether they could be used to more effectively target feedback. 

2. To see how useful the participating lecturers found the actual process of using a feedback 



checklist. 
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Figure 1 The essay feedback checklist 



Before you hand in your essay have you done the following? Please 


Y 


P 


N 


tick the appropriate box 









Addressed the question throughout the essay? 

tutor feedback 



Organised it clearly with structure appropriate to question? 

tutor feedback 



Put forward a relevant argument of good quality? 

tutor feedback 



Shown depth of understanding relating to underlying psychological 
issues? 

tutor feedback 



Evaluated theoretical concepts and research evidence? 

tutor feedback 

Referenced according to psychology requirements? 

tutor feedback 



Checked for spelling and grammar? 

tutor feedback 



Written in an appropriate academic style? 

tutor feedback | 

Are there any other feedback comments you would like your tutor to make 
about your essay?... 

Grade: 
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Results 

The descriptive analysis of the checklist findings are presented in Table 1 . Detailed inferential 
analyses are currently being prepared for publication elsewhere but basic analyses using t- 
tests are presented here. 

Total scores for all essay criteria: 

As can be seen, the totals at the bottom of the table show that both first and third year 
students thought they did better in meeting the assessment criteria than their lecturers did, t- 
tests showed that both these differences were statistically significant ( t = 4.54, p <0.0001 for 
first years and t= 9.73, p<0.000001 for third years). It can also be seen from these totals that 
the 3 rd year students thought they were better at meeting the criteria than the first year 
students and this was a statistically significant difference (t= -3.21, p<0.01). Interestingly, 
although it was not significant, there was some indication that the 3 rd year lecturers actually 
thought their students had done worse than the 1 st year lecturers thought their students had 
done. Perhaps this reveals higher expectation on the part of the lecturers teaching third year 
students. 

Individual scores for essay criteria: 

The initial finding to note is that the first year students thought they did better on all eight 
criteria than their tutors did, and all of these differences were statistically significant ( all 
probability levels were less than 0.05) except for ‘references’ and ‘academic style’. The two 
criteria where there was the biggest discrepancy were ‘put forward a relevant argument of 
good quality’ (t= 5.10, p<0.000001), followed by ‘evaluated theoretical concepts and research 
evidence’ (t= 4.27, p<0.0001). The argument criterion discrepancy supports other research 
which has shown this is one of the big areas of difficulty students get into as they simply do 
not understand what is meant by the concept of academic argument (see Hounsell, 1987). 
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Table 1 Means and Standard Deviations 
for students and lecturers on the essay feedback checklist criteria 





Year One 


Year Three 




Students (N=61) 


Lecturers (N=5) 


Students (N=65) 


Lecturers (N=2) 




Mean 


SD 


Mean 


SD 


Mean 


SD 


Mean 


SD 


QUESTION 
Addressed the question 
throughout the essay? 


1.75 


0.43 


1.48 


0.65 


1.82 


0.39 


1.32 


0.53 


STRUCTURE 
Organised it clearly with 
structure appropriate to 
the question? 


1.66 


0.48 


1.43 


0.56 


1.82 


0.39 


1.43 


0.66 


ARGUMENT 
Put forward a relevant 
argument of good 
quality? 


1.54 


0.53 


1.05 


0.53 


1.57 


0.53 


1.26 


0.48 


UNDERSTANDING 
Shown depth of 
understanding relating 
to underlying 
psychological issues? 


1.41 


0.62 


1.18 


0.56 


1.69 


0.47 


1.25 


0.61 


CONCEPTS/EVIDENC 

E 

Evaluated theoretical 
concepts and research 
evidence? 


1.51 


0.60 


1.03 


0.63 


1.52 


0.53 


0.86 


0.63 


REFERENCES 
Referenced according to 
psychology 
requirements? 


1.33 


0.63 


1.25 


0.62 


1.72 


0.52 


0.89 


0.77 


SPELLING 
Checked for spelling 
and grammar? 


1.92 


0.28 


1.69 


0.56 


1.92 


0.37 


1.22 


0.70 


ACADEMIC STYLE 
Written in an 
appropriate academic 
style? 


1.82 


0.43 


1.67 


0.51 


1.91 


0.30 


1.51 


0.56 


TOTALS 


12.93 


2.24 


10.77 


2.97 


14.05 


1.53 


9.74 


2.96 



ERIC 
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The second area where there was a large difference was in evaluating theoretical concepts and 
research evidence which is an area in psychology where students do have great difficulty 
particularly in their first year where they do not feel confident enough to challenge or question 
what they see in print. This finding supports King and Kitchener’s (1994) research on 
reflective judgement and confirms Perry’s (1970) model of intellectual development which 
shows that students tend to start with an absolutist view of knowledge and assume that it is 
unchallengeable, and the task is to absorb it from lecturers and other authorities and from 
other sources such as books, and the Internet. Challenging theoretical concepts therefore is 
likely to be particularly difficult in the first year. Evaluating research evidence is even more 
difficult for most students stick to textbooks and therefore only have secondary sources to go 
on so it is difficult for them to make any direct evaluation of a research study if it is cited in a 
book. We do encourage strongly, students to look at journal articles right from the start of 
their degree for this very reason, but it is usually only the more able students that will do this. 

The only two criteria where there were no statistical differences between tutors and students 
were in referencing according to psychology requirements and writing in an appropriate 
academic style. This suggests that, at the start of their academic degree, students are making 
big efforts to comply with the demands of the discipline. 

Looking at the third years, similar findings emerged with students thinking they had done 
better in meeting the assessment criteria than their tutors. As would be expected, the third 
year students also tended to rate themselves higher than the first year students on all the 
assessment criteria (except on spelling and grammar!). What was particularly worrying here 
was that the differences between students and their tutors were much bigger on all the 8 
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criteria and all these differences were statistically significant (probability levels were all less 
than 0.001). Of these differences, the smallest was for ‘putting forward a relevant argument 
of good quality’ which suggests that this is a concept that is understood better (but not 
sufficiently - the students rated themselves fairly low on this criterion) as students go through 
their degrees. 

Conclusions 

This research show quite clearly that there are discrepancies between what students think they 
do when they write essays and what their lecturers think they do. Worryingly, the discrepancy 
appears to increase by the third year which is not what we would hope! One explanation may 
be that the actual essay used in this research was the first one students had done in their third 
year and they simply had not adjusted to the much higher demands put on them at this level. 
Even though the criteria are the same for all years, there is a clear expectation in the 
department that students will be expected to become increasingly sophisticated in their 
writing as they proceed through the undergraduate programme. (An example is in the use of 
sources. In the first year we expect students to be mainly using textbooks but with the more 
able students perhaps consulting one or two journals. In the second year we expect the 
balance between books and journals to be more half and half but by the third year we expect 
our students to be going to the primary sources and using the journal papers themselves as 
their main sources, supplemented with text book information.) 
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Another possible reason for the bigger gap between lecturers and students in the 3 rd year 
compared with lecturers and students in the 1 st year may have been the conscious decision by 
the first year academic staff to encourage rather than criticise students in their academic 
writing at the very beginning of their undergraduate degrees. This speculation was lent some 
support in the interviews that the second author carried out with the lecturers, now briefly 
reported here. 

The interview 

Of the 7 Psychology lecturers who took part in the research, five agreed to be interviewed, 
their experience ranged from 3 years to 19 years. A semi-structured interview was carried 
out, using the schedule as shown in figure 2. 

The interview findings 

All five lecturers found essay marking difficult, particularly with borderline grades. Strategies 
ranged from comparing between students - norm rather than criterion assessment, to overall 
impression. Several of the lecturers mentioned problems of giving constructive feedback with 
poorer essays (a common problem with first years). 
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Figure 2. The interview schedule 



1. What experience have you had in marking essays? Do you find it easy/difficult? 

2. Is it made easier by the assessment criteria in Psychology? 

3. How did the feedback sheet compare? How did you find the process/can you recall how 
you used it? 

4. Having kindly participated in its use, did you find it useful? What improvements 
should/could be made? 

5. Do you think it should be taken on board as a departmental strategy? 

6. Our original idea was to interview students but time has caught up with us, have any of 
your students commented in any way on the sheet? 

7. Do you think this is a good idea to get them to focus on these criteria prior to handing their 
work in? 



All five agreed that the assessment criteria that are widely used in the department are useful, 
particularly in borderline cases. There was some indication that the lecturers did weight the 
different criteria - i.e. argument was mentioned as important by three of the lecturers. One 
lecturer mentioned that students did not always seem to understand what the criteria meant, 
hence the reason for this research. 



Three of the five found it interesting to see the discrepancy between the students’ judgements 
and their own, although one lecturer said it was difficult to baldly say ‘No’ as this was 
perhaps too harsh for first year students. Another lecturer wrote lots of comments against the 
tick box to justify her judgement. One lecturer made the very interesting comment that 
students might be judging themselves on effort. When asked about how useful they had found 
using the essay feedback checklist as part of the marking process, two said they found it 
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useful, one did not and the other two suggested a five point scale rather than a 3 point scale. 
More space for written feedback on the checklist was mentioned by 2 lecturers as a useful 
modification. 

The overall impression from the interviewees was that other colleagues might not agree with 
its introduction as a departmental policy but it might be worth doing if it could be shown to 
benefit students- three of the five suggested further research to see if it really was useful to 
students. 

Three out of the five thought asking students to focus on the criteria through the checklist 
probably would not help the students any more than the advice given in the first year’s 
handbook but they expressed an interest in seeing further research; two lecturers thought it 
was a good idea although one of the two thought it would probably only benefit the stronger 
students. 

Conclusions 

Basically the interviews with the tutors who had used the essay feedback checklist showed a 
moderate interest in further developing it but with the sensible reservation that students’ 
views should be sought. For the first years, it was thought that enough guidance was already 
given particularly in the First Year hand book (together with workshops on essay- writing 
using the criteria). 
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Overall conclusion from study 1. 

The evidence from both the statistical analyses and the qualitative findings suggest that using 
the essay feedback checklist is worth while developing, particularly by taking account of our 
students’ needs. 

Workshop presentation 

Essay feedback: How it can help students to improve their academic writing? 

Conference participants were presented with a brief account of research study 2 (see below) 
and invited to predict students’ responses to actual examples of lecturer feedback given in this 
research. Participants were given the opportunity to compare their predictions with the actual 
research findings. The session ended by drawing together issues from both the research and 
the workshop in a general discussion on the best ways of using feedback to help students to 
improve their academic writing 

Study 2: Effects of tutor feedback on student motivation in essay writing (Norton, 1997) 
Another explanation for why students fail to take account of feedback, may be linked to low 
self-esteem. Norton (1990) suggested that students’ self-esteem is linked with the 
coursework grades they are given and low self-esteem can be a powerful demotivator. 

This study was designed to see whether supportive and sympathetic tutor comments would 
help to minimise any negative effects of ‘low’ grades and thereby increase student 
motivation. In stage 1, 47 third year psychology students taking a third year option module in 
counselling psychology were given their essays back and then asked to indicate how useful 
they thought the tutor’s comments were. 
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Students were asked: 

Has the written feedback. . . 

Been useful to you? 

Motivated you to improve certain areas in your next counselling psychology essay? 
Increased your self esteem as a student? 



Students ticked one of four responses: 

Yes, definitely (scored 3) 

Yes, a little (scored 2) 

No, not much (scored 1) 

No not at all (scored 0) 

In Stage 2, 40 of the original 47 students took part. On the basis of the grades they got for 
their 1 st essay, students were systematically allocated to one of three feedback conditions for 
their next essay to see if the format of the feedback would have any effect. These 3 
conditions were 

1 . Follow-on: where feedback was given in two sections. The first section 
commented on improvements they had made from their 1 st essay and the 2 nd section 
suggested ways they could further improve for the next essay which was the exam. 

2. Student- request: where students wrote down what particular aspects of their essay 
writing they wanted comments on. 

3. Departmental criteria: where the standard departmental feedback sheet was given 
as explained in the study earlier. 

Then when the 2 nd essay was marked, students were once again asked to complete the same 
questionnaire in class time to indicate how useful they had found this second set of feedback. 
Results 

In order to see if it was the grade that had the greatest effect on how students perceived he 
feedback, correlations were carried out as shown in Table 2 
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Table 2 Correlations between essay grade and effects of lecturer feedback 



Grade for essay 1 
(N=47) and: 


r 


P 


Motivation 


0.18 


NS 


Self-esteem 


0.69 


0.0001 


Usefulness 


0.35 


0.01 




Grade for essay 2 
(N=40) and: 


r 


p 


Motivation 


-0.11 


NS 


Self-esteem 


0.52 


0.001 


Usefulness 


0.26 


0.05 



Conclusions 

As can be seen from this table, the greatest effect of the essay grade was on students’ self 
esteem and this was so in both essays. Students who got higher grades also perceived the 
feedback to be more useful, but there was no effect on motivation, so perhaps the effects of 
tutor praise are playing a part here. 



A basic content analysis of responses to the question about academic self-esteem was carried 
out which showed that the three factors mentioned most frequently in both essays were essay 
grade, tutor praise and positive effects on confidence. Additionally, there was some evidence 
to show that tutor praise had marginally more of an effect on students’ academic self-esteem 
than did the grade they were given. 



The findings have confirmed the author’s view that students are strongly affected by the grade 
they receive (Norton, 1990). A quote from a mature student’s counselling psychology journal 
is reproduced here with his permission. The student expected a grade somewhere between a 
C and a B, and actually got a C-. 
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" Today we received our Counselling psychology essays back. I knew I should have 
stayed in bed this morning. As soon as 1 saw my grade, all of the irrational beliefs I 
had been feeling after the physiology grade I had received in Part 1, came flooding 
back to me. 'Perhaps I shouldn t be doing this course ’ etc etc. At the moment I don ’t 
feel like writing in my journal, today ’s entry therefore will be a short one. All I can 
think of are negative thoughts; what am I doing at college at my time of life. Trying 
to hold down a part-time job, be there for my family and trying to study and complete 
course work is really getting me down. It must be so much easier to complete a 
degree course without the extra commitments of a family and a job. I know that if I 
pack the job in I will have a great deal more time for the other things in my life, 
however, finances wont allow this. ” 
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